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FREQUENCY BASED MODULATOR COMPENSATION 

BACKGROUND OF THE INVENTION 
[0001] Power amplifiers (PAs) are indispensable components in 
communication systems and are nonlinear in nature. At the PA output, the 
nonlinearity not only causes spectral regrowth, which interferes with 
adjacent channel signals, but also degrades the bit error rate of the inband 
signal. To compensate for the nonlinearity, PA linearization is often 
necessary. 

[0002] Among the linearization techniques, digital baseband predistortion 
is highly cost effective. It adds a functional block, called a predistorter, 
before the digital-to- analog conversion and RF upconversion but after any 
other baseband processing. The predistorter ideally applies the exact 
inverse response of the power amplifier to a scaled version of the input 
signal. Therefore, the output of the predistorter-PA cascade is the input 
signal multiplied by a scaling factor. To construct such a predistorter, a 
feedback path is needed to capture the output of the PA. 
[0003] The performance of baseband predistortion relies on accurate PA 
modeling and obtaining a precise inverse of the PA. In reality, however, the 
performance can also be affected significantly by imperfections in the 
upconverter in the transmitter and the downconverter in the feedback path. 
These imperfections are caused by the analog components employed in both 
the upconverter and the downconverter, such as mixers, filters, quadrature 
modulators, and quadrature demodulators. 
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[0004] To reduce these impairments in the downconverter, filters that 
have relatively flat frequency may be used, such as LC bandpass and 
lowpass filters, and digital demodulation, which is free of any demodulation 
errors. With these configurations and careful design, the imperfections in 
the downconverter can be negligible. 

[0005] In the upconverter, there are two common configurations, but the 
analog impairments normally cannot be neglected in either. The first uses 
digital modulation and two-stage upconversion, i.e., first upconvert 
baseband signals to IF and then to RF. Because of the stringent image 
rejection requirements of the transmitter, a SAW filter is usually used in the 
IF stage for this configuration. But the SAW filter often has large frequency 
response variations, and therefore distorts the predistorted signal. The 
second choice for the upconverter is to use direct upconversion, in which 
the I/Q data streams are directly modulated to RF. This structure enables 
the upconverter to be easily reconfigured to generate RF signals in different 
frequency bands. It also uses fewer components and is easier to integrate. 
However, in practice, the quadrature carriers in the analog modulator do not 
have exactly the same amplitudes and an exact phase difference of 90 
degrees. These effects are called gain/phase imbalance and cause cross- 
talk between the I and Q channels. In addition, leakage of the carriers to 
the transmitted signal manifests itself in the demodulated received signal as 
a dc offset. If uncompensated, the gain/phase imbalance and the dc offset 
would have a large effect on predistortion performance. Various techniques 
have been proposed in the past to compensate for these impairments. In 
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these techniques, the gain/phase imbalance are assumed to be frequency 
independent. However, experiments performed by the inventors indicate 
that the gain/phase imbalance exhibits frequency dependent behavior when 
the signal bandwidth becomes wide, for example, 15 MHz. 

SUMMARY OF THE INVENTION 
[0006] The present invention provides a transmitter that includes an 
upconverter for converting a lower frequency signal to a higher frequency 
signal and a compensator that compensates for at least gain/phase 
distortion introduced into the lower frequency signal by at least the 
upconversion. The compensator may also compensate for dc offset 
introduced into the lower frequency signal. 

[0007] In one exemplary embodiment, the upconverter is a direct 
upconverter that directly upconverts a baseband signal to the RF signal, and 
the baseband signal includes in-phase and quadrature phase components. 
The compensator includes filters compensating for gain/phase imbalance in 
the in-phase components and gain/phase imbalance in the quadrature 
phase components. 

[0008] In another exemplary, the transmitter includes a compensator 
constructor that, based on a model of at least the direct upconverter 
including an in-phase channel, a quadrature phase channel, and cross 
coupling channels between the in-phase and quadrature phase channels, 
estimates the in-phase channel, the quadrature phase channel, and the 
cross coupling channels between the in-phase and quadrature phase 



3 



Attorney Docket No. 29250-00 1081 /US 
Lucent Ref: 126019/Ding 2-1 1-4-17-12 

channels. The compensator constructor constructs the filters in the 
compensator based on the estimated channels. 

[0009] In another embodiment, at least one filter to correct for gain/phase 
distortion introduced by an upconverter is derived based on an inverse of 
the channel model for an upconverter. The inverse of the channel model for 
the upconverter is based on a cost function, which represents a mean 
square error, in the frequency domain, between a desired response of a 
system including at least the upconverter and an actual response of the 
system including at least the filter and the upconverter. 



BRIEF DESCRIPTION OF THE DRAWINGS 
[OOIOJ The present invention will become more fully understood from the 
detailed description given herein below and the accompanying drawings which 
are given by way of illustration only, wherein like reference numerals 
designate corresponding parts in the various drawings, and wherein: 
[0011] Fig. 1 illustrates the general structure of transmitter with a 
baseband predistortion system according to an embodiment of the present 
invention. 



DETAILED DESCRIPTION OF THE EXEMPLARY EMBODIMENTS 
Channel Models 

[0012] Fig. 1 illustrates the general structure of transmitter with a 
baseband predistortion system according to an embodiment of the present 
invention. As shown, a main signal path includes a predistorter 10, an I/Q 
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compensator 12, a digital-to-analog converter (DAC) 14, a direct upconverter 
16 and a power amplifier 18 connected in series. A coupler 20 couples the 
signal output from the power amplifier 18 to form a feedback path that 
includes a down converter 22, an analog-to-digital converter (ADC) 24, a 
digital demodulator 26 and a predistorter design unit 28. As shown in 
phantom lines, a secondary coupler 30 selectively delivers the output of the 
direct upconverter 16 to the down converter 22, and an I/Q compensator 
constructor 32 selectively configures the taps of the I/Q compensator 12. 
Except for the I/Q compensator 12, secondary coupler 30 and I/Q 
compensator construction unit 32, the other elements of the baseband 
predistortion system are well-known in the art. For the purposes of brevity, 
these elements will not be described in detail. 

[0013] As discussed in the background section, the down converter 22 has 
a relatively flat frequency response and the digital demodulator 26 is free of 
demodulation errors. Accordingly, the feedback path causes no extra 
distortion except for additive white noise. In the main path (i.e., the 
transmitter path), direct upconversion has been adopted, which may be 
readily reconfigured to generate RF signals in different frequency bands, but 
introduces gain/phase imbalance and dc offset. The additional baseband 
processing for the input signal u(n) in the transmitter includes predistortion 
and I/Q compensation, whose outputs are denoted by z(n) and x(n), 
respectively. 

[0014] During an initialization phase, the predistorter 10 and the I/Q 
compensator 12 are bypassed (i.e., x(n) = z(n) = u(n)) by having the I/Q 
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compensator constructor 32 output a training signal to the DAC 14. The 
secondary coupler 30 sends the output of the direct upconverter 16 to the 
down converter 22, bypassing the PA 18, during this initialization phase. 
Based on x(n) and the output y(n) from the digital demodulator 26, the I/Q 
compensation constructor 32 estimates the parameters of a channel model 
as discussed in detail below and determines the taps of the filters forming 
the I/Q compensator 12 as discussed in detail below. After the initialization 
phase, the bypassed elements are inserted and the predistorter 10 is trained 
in the well-known manner. 

[0015] As will be appreciated the initialization may be performed in many 
ways. For example, the predistorter 10 may be kept in the loop. Here, the 
secondary coupler 30 is not needed, as the output of the power amplifier 18 
is received by the down converter 22 via the coupler 20. Based on the input 
to the predistorter 10 and the output y(n) from the digital demodulator 26, 
the I/Q compensation constructor 32 estimates the parameters of a channel 
model as discussed in detail below and determines the taps of the filters 
forming the I/Q compensator 12 as discussed in detail below. 
[0016] During operation, the I/Q compensator constructor 32 may 
periodically or upon command re-estimate the received parameters using 
the compensation signal x(n) and the output of the digital demodulator y(n) 
and determine new taps of the filters forming the I/Q compensator 12. In 
this manner, the I/Q compensator 12 dynamically adapts to changes in the 
gain/phase imbalance and/or dc offset. 
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[0017] Next the method of modeling the channel creating the gain/phase 
imbalance and dc offset will be described in detail, followed by a description 
of estimating the parameters of the channel model. Lastly, determining the 
taps of the filters forming the I/Q compensator will be described. As part of 
the above-described aspects of the present invention, a detailed structure of 
the I/Q compensator according to one embodiment of the present invention 
will be provided. 

[0018] A detailed view of the channel from x(n) to y(n) , is shown in Fig. 2, 
where Re{ •} and Im{ •} denote the real and imaginary parts of a complex 
number, respectively. We also use subscripts i (in-phase) and q 
(quadrature) to denote the real and imaginary parts of a complex sequence; 
for example, we have 

x(n) = x, (n) + jx q (n), y(n) = y, («) + jy q (n) . 
[0019] Note that there may be a fixed delay and phase rotation (caused by 
the phase difference between the LOs) between x(n) and y(n), but they can be 
easily removed in any well-known manner. 

Real I/Q Channel Model 
[0020] The frequency dependent gain/phase imbalance affects the I and Q 
channels and the cross coupling between them. Moreover, the analog 
lowpass filters (LPF) and the mixers on the I and Q paths in Fig. 2 may not 
be exactly the same. To model the I and Q channels and the cross coupling 
channels between them, we use four real filters, h n , hi 2 , h 2 i, and h 22 ,, 
where hn and h 22 represent the I and Q channels, h i2 represents the cross 
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coupling of the Q channel with the I channel, and h 2 i represents the cross 
coupling of the I channel with the Q channel. The channel output y(n) then 
can be written as 

y(n) = g {[*,. (n - k)h, , (k) + x q (n- k)h, 2 (k)]+ j[ Xq (n - k)h 22 (k) + x, (n - k)h 2x (*)} 
+ d i +w i (n) + j[d g +w q (n)] 

where m(n) and w q (n) are, respectively, the real and imaginary parts of the 
complex white noise w(n) and d ( and dq are, respectively, the real and 
imaginary parts of the dc offset d. In (1), we assume that all four filters have 
the same length K. This assumption simplifies the model and the 
derivations in the following sections although the same methodology can 
still be applied if the h filters have different lengths. 



Complex I/Q Channel Model 
[0021] In (1), we can combine the terms that have xt(n-k) or Xq(n-k) and 
rewrite in a more compact form; i.e., 

y(") = Z k in - k)h t (k) + x q (n- k)h q (k)]+ d + w(n), (2) 

where 

h ( (k) = h u (k) + jh 2l (k), h q (k) = h, 2 (k) + jh 22 (k) . 
[0022] In other words, 

K x (k) = Re-fa. (k)}, h l2 (k) = Re{h q (k) }, 
h 2l (k) = hn{h i (k)}, h 22 (k) = lm{h q (k)\. 

[0023] Note that if there is no dc offset in the system and l^flc) =jhi(k), y(n) 
can also be written as 
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K-l 

y(n) = £ x(n - k)h(k) + w(n) , (3) 

k=0 

where h(k)=hi(k) = -jhq(k). In this special case, the model in (2) is simplified to 
a single complex filter with coefficients h(k). This is the case when a digital 
IF (intermediate frequency) modulator (a form of upconverter that converts a 
baseband signal to an intermediate frequency signal) is used instead of a 
direct conversion modulator or an analog IF modulator. Here, the error 
introduced by the IF modulator and subsequent RF upconverter for 
converting the IF signal to an RF signal is more broadly thought of as 
gain/phase distortion, which in the case where h(k)=Mk) = -jhq(k) does not 
hold true includes gain/phase imbalance. 



Direct/Image Channel Model 
[0024] The complex I/Q model in (2) gives the relationship between the 
system output y(n) and the real and imaginary parts of the system input 
x(n). However, it is not clear from (2) how the channel affects the input x(n) 
as a whole. We know that 



2 * 2j 



(4) 



where (•)* denotes complex conjugate. Substituting (4) in (2), we obtain 



AT-l 



y(n) = 2 



<t=o 



x(n-k) + x*(n-k) 



2; 



+ d + w(n) . (5) 



Rearrange the r.h.s. of (5) to write 
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K-l 



y(n) = Z [*( n ~ (*) + **(»- k)h m (k)] + d + w(n) , 



(6) 



where 



h d (k) = 



h,{k)-jh 9 (k) 



h m (k) = 



h t {k-) + jh q {k) 



2 



2 



are, respectively, the direct and image transfer function. Next we look at the 
relationship between x(n) and x*(n). If x(n) is a lower sideband signal, it can 
be expressed as the summation of a real sequence xa(n) and its Hilbert 
transform x d (n), i.e., 



[0025] Since Xd(n) is a real sequence, it has a double sideband frequency 
spectrum Xdef 0 ). The spectrum of x(n) is actually the lower sideband of 
Xdfef") but with twice the amplitude. From (8), it follows that 
x*(n) = x d (n) - jx d (n) , whose spectrum is the upper sideband ofXd(e) w ) with 
twice the amplitude. Similarly, when x(n) has an upper sideband spectrum, 
the spectrum of x*(n) is located at the lower sideband. For this reason, we 
caH x*(n) the image of x(n), hm(k) the image channel transfer function, and 
Mk) the direct channel transfer function. In the channel model (6), we can 
see that if hi(k) * hq(k), i.e., run(k) * 0, and the input x(n) has one-sided 
spectrum, the image of x(n) will show up in the other side of the spectrum, 
and its amplitude is determined by hm(kj. When the input x(n) has a double- 



x(n) = x d (n) + jx d in). 



(8) 
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sided spectrum, the images of lower and upper sidebands of X(^°) will also 
appear in the spectrum of the output; however, the images may be 
completely covered up by the spectrum of the original input signal in that 
case. 

[0026] Note that the three models proposed are all equivalent models. 
However, they reveal different aspects of the channel, and each has their 
own pros and cons. 



Least-Squares Method of Channel Estimation 
[0027] For a block of Nx(n) and y(n) data samples, (2) can be written 
vector form; i.e., 

y = Xih t + Xqhq + dip + w, (9) 
where y = \y(K-L— 1) y(K - L) . . . y(N - L - l)f is a vector length 

P = N-K+\ (10) 



m 



with L a selectable delay, X, = Re(X) and Xq = Im(X) with 



X = 



x(K-l) x(K-2) x(K-3) ••• jt(0) 
x(K) x(K-l) x(K-2) ••• jc(1) 

x(N-l) x(N-2) x(N-3) ■■■ x(N-K) 



(ID 



hi = [h t (0)/i, (1) • • • h, (k -l)f,h q = [h q (0) h q (1) • • • h q (K - 1)] 7 " , 1 P is a column vector 
flUed with all ones, and w = [co (K- 1) co{K) ... w(N - 1)]t Since Kis the length 
of the model channel filter, we choose 

L=L(^-l)/2j, (12) 
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where |_*J is the largest integer that is less than or equal to x, such that the 
maximum response in the model channel filter is located approximately at 
the center tap. Note that not all N samples of y(n) are used in the 
formulation in order to avoid the boundary effect. The system output y(n) is 
nominally matched with the input x(n), i.e., the relative delay, amplitude, 
and phase difference between y(n) and x(n) have been removed. 
[0028] To estimate the channel coefficients, we define a cost function as 
the mean square difference between the actual output y and the noiseless 
model output: 

J = (y - Xih - Xqh q - dip)" (y - Xih - Xqh q - dl P ), (13) 

where (•)« denotes Hermitian transpose. The optimal hi, h q , and d that 
minimize the cost function can be found by setting the partial derivatives of 
J with respect to h*,h* q , and d* to zero (assuming that h, hq and d are 
constants; i.e., 

dJ T = X? (y - Xfo - X q h q -dl P ) = 0 (14) 



BJ 



- = X H q (y - Xfy - Xh -dl p ) = 0 (15) 



Bh* q " q " q " q 



BJ 



- = l T P (y- X,A - X q h q - dl P ) = 0 (16) 



Bd* 
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[0029] Since Xi and Xq in (14) and (15) are real matrices, the Hermitian 
transpose can be replaced with a transpose without conjugate. Rearranging 
(14) - (16) and putting them in a single equation, we have 

XjX : XjX„ Xj\ 

(17) 



l T P X t 



i T P x q 



x T q i P 

1 T P 1 P 







'Xjy 


\ 




Ky ■ 


d 




i T P y 



[0030] Therefore, the least-squares estimates of the hi, h q , and d are 



~h~ 




xjx t 


xjx q 


xj\ p 


-l 


'xjy 


K 












Ky 


d 




l*X, 











(18) 



[0031] Note that a coarse estimate of the dc offset can be obtained as the 
difference between the mean of the system output and that of the system 
input; i.e., 

d=^Y d [y(n)-x(n)]. (19) 



where N is the window of samples over which the dc offset is determined. 
From the complex I/Q model, we know that d is given by 



d = d„ 



ReR E K (k) + Jm{d x }§ h q (*) 

k=0 



(20) 



where d x and dy are, respectively, the mean values of the input x(n) and the 
output y(n). Therefore, in order for (19) to be accurate, either d x needs to be 
very small or ^™h t (*) « 1 and X*=~<X <*> " J ■ If either of these conditions 
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hold, then we may use (19) to estimate the dc offset and adjust the cost 
function (13) to 

J = (y - d\ P - X,h, - X q h q )"(y-dl p - Xfy - X q h q ) , (21) 

[0032] Similar to the previous derivation of the least-squares solutions, 
the least-squares estimates of the hi, h q are then given by 







~xjx t xjx; 


-1 




-dl P )~ 


s_ 




x T q x t x T q x q _ 






-di P ) 



Least-Squares Method with Diagonal Loading 
[0033] Since the input signal x(n) is usually a bandpass signal, the 
channel estimates are only accurate within the band. The out-of-band 
responses are solely determined by the wideband noise in y(n). These 
arbitrary out-of-band responses are not desired in predistortion 
applications, where the low level out-of-band signal needs to be accurately 
preserved in order to compensate for the PA nonlinearity. A simple solution 
is to add an artificial white noise to both x(n) and y(n) to create a flat 
frequency response over the whole band. The artificial noise level may be 
controlled to be well above the original white noise level in y(n) but well 
below the inband signal level. Thus, an artificial flat out-of-band response is 
created that preserves the inband channel response. Let us denote the 
artificial white noise as v(n), whose real and imaginary parts are, 
respectively, Vi (n) and v q (n). Similar to the definition of Xt, and Xq, we can 
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define Vi and V q and rewrite (18) for the case where v(n) is added to both x(n) 
and y(n), which leads to 

\x,+v,) T ix,+v,) (x l +v i ) T (x q +v q ) (x,.+v;.) r i p 

(X q+ V q ) T (X i+ V i ) (X q+ V q ) T (X q+ V q ) (X g+ V g ) T l f 

ip(x,.+v;.) i T P (x q+ v q ) i T p i p 



x 



iX t +V,) T (y + v) 
(X q +V q ) T (y + v) 
lp(y + v) 



(23) 



where v = [v (K-l) v(K) ... v(N-l)]T Ideally, u f (n; and v q (n) are uncorrelated 
with each other and are uncorrelated with x(n) and y(n). Moreover, the mean 
value of v(n) is zero. Therefore, instead of calculating (23), we may estimate 
the channel parameters using the following formula 



X/X +CT 2 I 


xjx q 


xj\ p ~ 




~Xjy + a 2 e 


x T q x t 


x T q x q+( j 2 i 


x T q i P 




X T q y + a 2 e 


l T P X t 


i T P x q 


l T p l p 




i T P y 



where a 2 is the variance of the artificial noise, I is a K x K identity matrix, 
and e = [o[ 1 0 T L J , where 0 L and 0 M are, respectively, length L = [_( K - 1) / 2j 
and M = K - L - 1 column vectors filled with all zeros. In the case where the 
dc offset is estimated using (19), the least-squares estimates of the hi, h q 
with diagonal loading reduce to 



XjX t +<x 2 l 



xjx. 



X q X q +a 2 I 



Xf(y-dl P ) + a 2 e 
X T q (y-dl P ) + a 2 e 



(25) 
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[0034] In either case, the diagonal loading has the additional advantage of 
regularizing the solution, i.e., reducing the condition number of the 
correlation matrix, so that more accurate solutions can be achieved. 
[0035] As will be appreciated from the above description, the I/Q 
compensator constructor 32 estimates the dc offset from the signals x(n) and 
y(n) using equation (19) and estimates the response of the I or in-phase 
channel h t and the response of the Q or quadrature phase channel it from 
the signals x(n) and y(n) using either equation (22) or equation (25). 

Construction of the I/Q Compensator 
[0036] After obtaining the channel filter estimates h t and h q and the dc 

offset estimate d , the I/Q compensator 12 is constructed by the I/Q 
compensator constructor 32 to compensate for these imperfections. It turns 
out that these imperfections can be fully compensated by an I/Q 
compensator that has structure similar to the channel models described 
above. To ease the derivation, one exemplary embodiment uses the real I/Q 
channel model to represent both the I/Q compensator 12 and the baseband 
direct upconverter 16. A block diagram of the cascade of the I/Q 
compensator 12 and the upconverter 16 is shown in Fig. 3. As shown, the 
I/Q compensator 12 includes four real finite impulse response (FIR) filters, 
gn, gi2, g2i, g22. The first filter gn filters the in-phase components and the 
second filter g i2 filters the quadrature phase components. A first adder 50 
adds the outputs of the first and second filters gn and gi 2 to produce a 
gain/phase compensated in-phase signal. The third filter g2i filters the in- 
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phase components and the fourth niter g22 filters the quadrature phase 
components. A second adder 52 adds the outputs of the third and fourth 
filters g2i and g2 2 to produce a gain/phase compensated quadrature phase 
signal. A third adder 54 adds an in-phase dc component q to the in-phase 
signal and a fourth adder 56 adds a quadrature phase dc component a, to 
the quadrature phase signal to compensate for dc offset. 



Offset Compensation 
[0037] Let us first consider the compensation of the dc offset d using the 
dc component (q, c,) in the I/Q compensator 12. After passing through the 
four channel filters gn, gi 2 , g2i, and g2 2 , the dc component c becomes 



c f =c,s u +c g s 12 

c =c s + r v ' ( 26 ) 



where 



*n=2X(*). *i2=Zm*) 

a • (27) 



*=0 <t=0 



[0038] Thus, adding c in the I/Q compensation 12 is equivalent of adding 
a dc component c at the end of the cascade. In order to compensate for the 
dc offset d , we would like to have c i = and c = -d ; i.e., 
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It is clear from (28) that the solution for a and Cq is 



V 




S ll s \2 


-1 




_v 




_ S 2l S 22_ 




-d 

- 1 - 



(29) 



[0039] Note that when the Inverse matrix in (29) is close to the identity 
matrix, we have 

Cq * - d ■ (3 °) 

[0040] Accordingly, the I/Q compensator constructor 32 sets the dc offset 
compensation a, Cq according to equation (29) or (30). 



Gain/Phase Imbalance Compensation 
[0041] Next we consider the problem of designing the four filters gi i , g l2 , 
g2i, and g2 2 in the I/Q compensator 12. We adopt a frequency domain 
approach to solve this problem. Denote the Fourier transform of yi (n) and 
y q (n) by Y i (d eo ) and Y q (e>°), respectively, and the Fourier transform of Zi (n) and 
ZqfnJ by Z{en and Zqiei 0 ), respectively. The outputs Yt(eP°) and Y^e^ of the 
digital demodulator 26 can be expressed as 









- 



H u (en H l2 (en 
lH 2l (e Ja >) H 22 (en 



G u (en G 12 (e J ») 
G 2l (en G 22 (e J ») 



z^en 

L Z q (e JO> ) 



(31) 



To make YlfeO = Z(e> M ) and Y q (^°) = Z4&% we need 
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G u (en G l2 (e j<0 ) 
G 21 (e^) G 22 (e J *)] 



H n {e i0 >) H l2 (en 
H 2l (e Jto ) H 22 {e ]a >)\ 



H 22 (en -H l2 (en 
-H 2l (e Ja >) H n {e itt> ) _ 



where 



A(e J ) = t - i 

[H n {enH 22 {e ia )-H n {e^)H 2l {e^)\' 

[0042] In the time domain, we may view (32) as follows, 

g n (k) = h 22 (k) * a(k), g n (k) = -\ 2 (k) * aik) 
g 2i (k) = -h 2i (k) * a(k), g 22 (k) = ^ (k) * a(k), 



(32) 



(33) 



(34) 



where * denotes convolution and a(k) is the inverse Fourier transform of 
AfeJ 03 ). From the definition of AteH, a(k) is the inverse of filter 

h c (k) ~ h 22 (k) ~ L (k) * h 2l (k) ; (35) 

i.e., the convolved response of filters a(k) and hc(k), denoted by c(k), 
approximates a delta function with a certain delay. In other words, we 
would like the Fourier transform of c(k) to be close to a desired frequency 
response e~ Jan ° , where no is the desired delay. To find the optimal a(k) 
coefficients, we suppose that filter afk) has Ka taps, the convolved response 
of a(k) and hc(k) can be written as 

c(Jt)= 2lh c (k-l)a(l),k = 0,l,--,K a +2K-3, (36) 

which has length Kc = Ka + 2K - 2. [Note that filter hc(k) is of length 2K- 1]. 
We then define a cost function as the integrated difference between the 
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frequency response of filter c(k) and the desired frequency response within 
given passband [~co p ,co p ]\ i.e., 



K r -l 



*,=0 



K c -\ 
_k 2 =0 



dco . (37) 



[0043] Substituting (36) into (37), we have 



i.-r 



_k x =0 / 1= 0 



K c -\K a -\ 

X 2 K (k 2 - Z 2 )a(l 2 )e~ j ^ - 
t 2 =o; 2 =o 



(38) 



[0044] To find the optimal a(k) that minimizes the cost function, we take 
the partial derivative of J c in (38) with respect to a*(l) (assuming a(l) is 
constant and set it to zero); i.e., 



dJ 



L /t,=0 ^=0 



AT C -1 



x Z*«<*2 -1)^^ = 0; Z = 0,...,^-1. 



* 2 =0 



[0045] Rearrange (39) to write 

** (0 (,=o t,=o*lTo 



V*; (* 2 - 1) r e-^-^dm = 0 



(39) 



(40) 



[0046] In predistortion applications, the least-squares with diagonal 
loading is used for estimating the four h filters, and filters hu(k) and h22(k) 
have flat responses outside of the signal band. Moreover, the responses of 
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the cross coupling filters hi 2 (k) are usually much smaller than those of the 
filters hi\(k) and hz2(k). Therefore, the out-of-band response of filter hc(k) 
may be considered as flat with unit gain. Thus, we can use co p = ti in (40). 
In the channel estimation phase, we want the maximum amplitudes of the 
filters to occur around the center taps. When we construct the inverse 
filters here, we also let our desired response for filter c(n) be a delta function 
around the center tap; i.e., no = (Kc - D/2. Substituting a) p =n no = (Kc - 
1)11 into (40) and carrying out the integration, we obtain 

3/ f.-l K c -\K c -\ 

JT±Z = 2><'i > Z 2>c (*. - Z i K (*2 - 02tt sine (*, - k 2 ) 

-2LK{k 2 -l)27r sinc(n o -£ 2 ) = 0, (41) 
where 

sincfo/ = . (42) 

7DC 



[0047] For I from 0 to Ka - 1 , we have a set of equations from (41); i.e., 

K a -\ K C -\K C ~\ 

Z a Qy ) Z Z 2n K (*i ~ A )>C (* 2 - '2 ) sine (*, - k 2 ) 

;,=o i,=ot 2 =o 

= Z 2n K (*a - 0 sine (*! - * 2 ) . (43) 

*,=o 



[0048] Removing 2^ from both sides of (43) and rewriting it in matrix 
form, we have 
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R h a = h a (44) 
where a = [a 0 ...a K _ x Y , and the elements of matrix Rh and column vector h a 
are defined as 

K —IK — 1 

«*('i.0= EE KiK -h)K(k 2 -l) sincC/:, -k 2 ) (45) 

*,=0* 2 =0 

K(V= T,K (k 2 -l) sine (n 0 - A: 2 ) . (46) 

k 2 =0 

[0049] Note that Rh is a Toeplitz matrix, so we only need to calculate the 
first row and column of the matrix. The least-squares estimate of a from 
(44) is 

« = *a~V (47) 

[0050] Accordingly, the I/Q compensator constructor 32 determines a 
according to equation (47) and then determines the filter taps for filters gn, 
gi2, g2i, and g22, according to equation (34) using a as a{k). 
[0051] The output from the power amplifier 18 is then transmitted in a 
conventional manner. Any well-known receiver for receiving transmitted 
signals may then receive and demodulate the transmitted signals. For 
example, the receiver may have the structure of the feedback path shown in 
Fig. 1. 

[0052] In an alternative embodiment, the filter taps for the filters gn, gi 2 , 
g2i, and g22 are directly estimated using equation (18) or (24). In using 
equations (18) and (24), y(n) is treated as the input and x(n) is treated as the 
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output. The resulting hi, h q , and d then form the basis for the I/Q 
compensator 12. 

[0053] The invention being thus described, it will be obvious that the 
same may be varied in many ways. Such variations are not to be regarded 
as a departure from the spirit and scope of the invention, and all such 
modifications are intended to be included within the scope of the following 
claims. 
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